Motivation and context:
Robust monitoring, evaluation, and learning (MEL) frameworks can play an essential role in increasing one's
adaptive capacity and resilience to climate change, by allowing organizations, policymakers, and practitioners
to assess and improve the effectiveness, efficiency, and equity of their climate change adaptation actions
regardless of the context and scale.
This project and report's specific purpose was to identify
relevant MEL frameworks, tools, and approaches that could be applied to systematically measure the impact
of knowledge products on the implementation and scaling up of adaptation in the context of the work of the
NWP.
Methodology
The research and writing team of this report adopted a stepwise approach and methodology that consisted
of four sequential steps:
- a review of the existing literature,
- soliciting inputs from experts through
interviews,
- a comparative assessment of relevant MEL approaches, frameworks, and tools, and
- formulating recommendations.
The insights from the literature review, expert interviews, and comparative
assessment were iteratively integrated in this report, initially discussed in different sections, to then be
combined to form the evidence base for the recommendations. The literature review consisted of both peer-reviewed
literature and gray literature – including, amongst others, reports and working papers from
governments, NGOs, and international organizations. Semi-structured interviews based off a standardized
questionnaire constructed by the research team were conducted with various experts engaging in the field of
MEL, who shared insights according to their knowledge and experience in applying specific MEL approaches
and tools to their work.
The comparative assessment, divided the approaches and applications found in the
literature review and expert interviews by shared features, resulting in three distinct overall approaches of
MEL to be compared. Relevant applications of each approach were then described and comparatively
assessed over a variety of criteria.
The three initial phases of the project culminated in a series of evidence-based
recommendations and MEL framework features that are highlighted and divided into 3 main subcategories.
Key findings
From the review of the literature, it emerged that there is no universally accepted definition of MEL in the
context of climate change adaptation. Key specific features such as indicators and criteria for assessment
differ according to the context, requirements, and objective of the different MEL frameworks, highlighting a
no ‘one-size-fits-all’ approach. However, some common features identified underscore the use, function and
importance of MEL frameworks, including integration of information across sectors, geographical scales, and
through time; community learning and capacity development; supporting governments in planning and
decision-making processes; and identifying investment priorities at different levels. Overall, across scales and
contexts, MEL systems are characterized by (1) the definition of the context of the MEL system; (2) the
identification of the content (i.e., adaptation intervention) to be monitored; (3) the design of the
operationalization process; and (4) the establishment of strategies to communicate the results, in line with
the purpose of the MEL system.
The comparative assessment of MEL approaches was articulated around three overall approaches
identified:
- Outcome Mapping which focuses on the changes - in behavior, relationships, activities or
actions of the ‘boundary partners’;
- Theory of Change/Theory of Action which helps to assess the
pathway through which action and change will be achieved; and
- Co-Productive Approaches with MEL
framework embedded in an inclusive, collaborative and flexible process to improve the use and uptake of
knowledge, leading to action.
Some key features highlighted in the assessment included the scope each
MEL frameworks was applied within (local, regional, national and global scales), the diversity of boundary
partners, the diversity in types of indicators used across different contexts and levels, and the types
adaptive, flexible and iterative approaches used in each framework. Several challenges identified across
approaches and applications included achieving sufficient engagement of key stakeholders across scales
with respect to inclusivity, capacity, and depth of engagement (amongst other factors) in the design and
operationalization of these MELs; the availability and accessibility of consistent, reliable and action-oriented
data to knowledge partners; and contextualization (or lack thereof) of adaptation outcomes and impacts
following an intervention action or project. Additional challenges found within all three initial phases of this
project address additional issues with effective and efficient stakeholder engagement, issues in navigating
limited resources (such as time, data, financial, technical, and/or institutional capacity), the impact of
climate change uncertainty factors, the impact of and difficulty in establishing baselines and targets for
future projects and projections, challenges in assessing attribution, and effectively applying an MEL
framework that can operate over long time horizons.
Recommendations:
The recommendations of this report are divided into three main categories:
- Methodological
Considerations,
- Indicators and Criteria, and
- Capacity Building and Institutional Context, all of which are
further supplemented by Appendix 05 which is comprised of a list of resources deemed to be relevant
references for the deployment and development of an operational MEL.
The Methodological
Considerations section is a series of recommendations on how to combine different approaches and
methodologies from the Theory of Change, Outcome Mapping, and Co-Productive MEL approaches for the
creation of an operational and specialized MEL framework for climate adaptation and knowledge uptake.
The primary recommendations of the Indicators and Criteria section revolve around the importance and
necessity of standardizing the operational definitions of key terms, concepts, and indicators across
programs and projects.
This includes the importance of including relevant climate indicators (such as
hydro-meteorological indicators) in projects revolving around climate adaptation or mitigation projects as a
factor for assessing the outputs, outcomes, and impacts under shifting baselines due to climate change
uncertainties.
The final recommendation section, Capacity Building and Institutional Context, highlights the
importance of designing and implementing a parallel structure within an MEL so as to assess both the
internal and external capacities of a project or program and the context (i.e., region, community,
institution, government) it is applied within to determine both the framework and project’s ability to
sustainably function and achieve set objectives. If the design of a project or program cannot be sustainably
supported or realized in the region, then that has implications for how successful and effective that project
and intervention may be. Both this section and the first include recommendations for the inclusion of and
amplification of stakeholders’ concerns, voices, and input.
The report is then concluded with a section discussing its report’s Limitations in the context of the course
framework, time frame, and resources available, followed by the bibliography and appendices.